Boosting boostrap FLD subspaces for multiclass problem

In this paper an ensemble feature extraction algorithm is proposed based on Adaboost.M2 for multiclass classification problem. The proposed algorithm makes no assumption about the distribution of the data and primarily performs by directly selecting the discriminant features with the minimum training error, which can overcome the main drawbacks of the traditional methods, such as Principle Component Analysis (PCA), Fisher Linear Discriminant Analysis (FLD) and Nonparametric Discriminant Analysis (NDA). The proposed method first samples a large number of bootstrap training subsets from the original training set and implements FLD in each subset to get a large number of bootstrap FLD projections. Then at each step of Adaboost.M2, the projection with the minimum weighted K Nearest Neighbor (KNN) classification error is selected from a pool of linear projections to combine the final strong classifier. Experimental results on three real-world data sets demonstrate that the proposed algorithm is superior to other feature extraction techniques.